Some major commercial Web services (e.g., e-retailers, search engines, etc.) have a platform consisting of a multitude of services (e.g., over 200) that are combined to build their web applications. They produce a software developer's kit (SDK) to allow third parties to create new applications using those services (through web protocols). Commercial Web services want third parties to do so because it drives traffic, advertising, and/or sales into the commercial Web services. The third parties want to build these applications because they receive money from the commercial Web services for each sale they drive.
The problem is that the commercial Web services have no good way of hosting the third party's code within their platform. With existing technology, the 3rd party hosts their own web site. This is bad for the third party because they must build a web site, acquire servers and network connections, and manage and operate their servers (even if they pay someone to host their servers). Depending on the specific scenario, this can be a very complex task requiring expertise in networking, security, distributed systems, system scaling, and so forth. This situation is bad for the commercial Web services because it creates a barrier that discourages new third parties from developing extensions to their platform.
The commercial Web services do not host the third party's code because there is no cheap and secure way to do so. The commercial Web services' conventional choices to host the third parties code are basically: put it on its own server (very expensive), put it in its own software-based virtual machine using a virtual machine monitor (VMM) like VMWare™ or Virtual Server (pretty expensive), or trust it (insecure).
Even with the first two options, the third party's code has access to a full network stack, so the commercial Web services must put firewalls (potentially expensive stateful firewalls) around it to keep it from communicating inappropriately with services' servers or to keep it from performing inappropriate actions, such as sending unsolicited spam to the outside world. Another impractical solution is for the commercial Web service to run the third party code a language-based virtual machine, such as Sun's Java Virtual Machine (JVM) or .NET AppDomains. While much cheaper than physical hardware or VMM solutions, the language-based VM solutions are inflexible, severely limiting the structure of third party code that can be safely run. Also, language-based VM solutions sacrifice security by enlarging the vulnerability surface and because they cannot adequately constrain resource usage.
The commercial Web services have no conventional approach that allows them to cheaply and securely host third party code in their data centers using the services' operations expertise. The independent software venders (ISVs) have no conventional approach that gives them the flexibility to write a new extension as dictated by their design needs and easily drop it into the commercial Web services' servers, securely and cheaply.
Using only the available conventional approaches, a developer who creates or extends an existing web application with code must become well versed in the expertise of running a data center. The developer must endure the hassle of server procurement, server management, patching, operations planning, service deployment, load balancing, backup, etc.
For example, let's say a fictional company, which we will call the Hope Software Corporation (HSC), wanted to extend a service called the World Mapping Web Service (WMWS). HSC wants to extend the WMWS service by plotting houses from various web sites and the multiple-listing services (MLS) real estate listings onto a satellite map from WMWS. Assuming that WMWS exposes the necessary application programming interfaces (APIs) to build this application, HSC still has to handle the operations burden imposed by being a web application. Say HSC's application is really popular and gets featured in an extremely popular web site. Now, HSC has to make sure their application is scalable to handle the extra load which can occur in a matter of hours. HSC has to make sure their application gracefully fails over in case one of their machines dies. HSC knows that if they cannot service their customers 24×7, someone else will.
For another example closer to home, let's say Bob modifies a service called Sharing Documents Web Service (SDWS). Bob modifies SDWS with a bit of workflow logic to route documents for approval in his workgroup. Bob's workgroup now has to worry about the hassle of obtaining a server, deploying the operating system and SDWS, patching, backing it up, etc. just because of a little extra customization. And the IT department is not completely off the hook anyway. There's now another one-off deployment of SDWS floating in the enterprise and any lapse in management of that server poses extra security risks and support costs.
In still another example, say HSC could modify (herein “mod”) a massively multiplayer online game (MMOG) like Everquest™ or World of Warcraft™. In doing so, HSC creates its own zones, monsters, and artificial intelligence (Al). Despite the huge success of mods for client-side games, mods for MMOGs are nowhere to be seen because the MMOG operators do not have a secure and reliable mechanism to isolate mods on MMOG servers.
There are an endless number of scenarios where developers want to extend existing web services. Even when a network-based interface to the web service is exposed, extending a web service is expensive and difficult because the programmers developing the extension are forced to become service providers in addition to developing the code.
There are of course hosted data centers where a third party can rent managed servers and run their code. But in this case, the third party essentially gets just a hardware box. The third party is not extending an existing web service such as one for online advertising, collaboration, business processing, gaming, etc. The third party still needs to worry about service deployment, load balancing, backup, etc. And there is still the cost involved. If HSC just wants to try out an idea, the cost just for hosting can be nearly a thousand dollars a year to use a service provider.
Thusfar, we have only described Internet scenarios. It is worth explicitly pointing out that the techniques described here are applicable to wide range of computation scenarios. For example, the “web service” that needs to be extended may actually be any arbitrary computation node, such as a mobile phone device or personal computer. It could even be an arbitrary computation system, such as a peer to peer or grid network. Consider a project like the SETI@home distributed processing system for analyzing radio signals. In this setup, each the owners of each PC participating need to “trust” the SETI@home software to not be malicious. However, this software is relatively fixed—it is not possible for an arbitrary astronomer to for example, quickly use the resources of those thousands of computers to evaluate a new radio signal analysis algorithm.
This scenario is exactly analogous to the web services scenarios described earlier, only in this case, the “commercial web service” is actually a distributed computation grid. Either way, it can be modeled as a web service needing extensions.